1. Field of the Invention
The present invention relates to a field detection apparatus and method, an image coding apparatus and method, a recording medium, a recording method and a transmission method. More particularly, the present invention relates to a field detection apparatus and field detection method, an image coding apparatus and image coding method, a recording medium and recording method and a transmission method, wherein an image signal obtained by pulldown processing a motion-picture film or the like is coded with the same fields deleted, and is recorded onto a storage or recording medium such as an optical disk, a magnetic disk, and a magnetic tape, and is transmitted via a transmission line.
2. Description of the Related Art
A motion picture projected in a movie theater is made up of consecutive photographs on a film. To broadcast it as a television program or to sale it in the form of a video package, the motion-picture signal needs to be converted into an image signal (video signal) that is an electric signal for television broadcasting. More particularly, the images on the film that are optically projected are electrically picked up, with the conversion of the display rate of image and other processings involved. Such a technique is generally called telecine.
The image signal and film are different in the number of images displayed per second. In many of motion-picture films, 24 pictures (frames) per second are projected while 29.97 frames are presented in the image signal in accordance with the NTSC standard. The conversion of frame rate is one of the major processings in the telecine. A technique called 2-3 pulldown is widely used to convert the frame rate.
A moving picture is made up of a set of consecutive images, each called a frame. The frame may be divided into fields as shown in FIG. 26. The frame constructed of odd-numbered line pixels is called a top field while the frame constructed of even-numbered line pixels is called a bottom field.
The frames are classified into two types: progressive pictures and interlaced pictures. The progressive picture and interlaced picture are different in the timing of sampling the image. More particularly, the top field and the bottom field are at the same sampling timing (FIG. 27A) in the progressive picture while there is a difference in the sampling timing between the even-numbered lines and the odd-numbered lines within one frame in the interlaced picture. As a result, the top field and the bottom field are different in the sampling timing (FIG. 27B).
The motion-picture film signal presents a progressive picture while the image signal presents an interlaced picture.
In the 2-3 pulldown process, the film of 4 frames is converted to the image signal of 5 frames. In the 2-3 pulldown process, one film frame is divided into the top field and the bottom field as shown in FIG. 26. The top field and bottom field are appropriately assigned to the fields of image signal.
More particularly, the four film frames are designated A, B, C and D in time series, and the top field and bottom field divided from the frame A are designated A(1) and A(2), respectively. Frames B, C and D are designated in a similar fashion. As shown in FIG. 28, a first frame is constructed of the top field A(1) and bottom field A(2), and a second frame is constructed of the top field B(1) and bottom field B(2). A third frame is constructed of the top field B(1) and bottom field C(2), and a fourth frame is constructed of the top field C(1) and bottom field D(2). A fifth frame is constructed of the top field D(1) and bottom field D(2).
As described above in the 2-3 pulldown processing, the field B(1) is assigned as the top field in a second frame and the subsequent top field (the top field in the third frame) that follows two fields behind the second top field. The field D(2) is assigned as the bottom field in the fourth frame and the subsequent bottom field (the bottom field in the fifth frame) that follows two fields behind the fourth bottom field. In this way, the film frames in the unit of four frames are converted to an image signal in the unit of five frames.
In the 2-3 pulldown processed image signal (the image signal obtained by 2-3 pulldown processing the film), a subsequent field that is identical to a field that precedes two fields ahead of the subsequent field appears every five fields.
In the 2-3 pulldown processed image signal as described above, a subsequent field, which follows two fields behind a field of interest and which is identical to the field of interest, is called a repeat field.
It has been recently proposed that the repeat field be removed to form a progressive picture before coding when the 2-3 pulldown processed image signal is coded in accordance with MPEG (Moving Picture Experts Group) standard. Since the repeat field is identical to the field that precedes two fields ahead of the repeat field, the removal of the repeat field reduces redundancy and improves coding efficiency.
To delete the repeat field, the detection of the repeat field from the 2-3 pulldown processed image signal is first required.
The inventor of this invention has disclosed a technique for the detection of the repeat field from the 2-3 pulldown processed image signal, for example, in U.S. patent application Ser. No. 08/578,317 (filed Dec. 27, 1997), U.S. Pat. No. 5,691,771.
The detection method of the repeat field proposed in the above disclosure is now briefly discussed.
In the detection of the repeat field, a differential value between a field of interest and a subsequent field that follows two fields behind the field of interest (the subsequent field comes in two fields later in time, namely a future field) is computed, and the differential value is compared with a predetermined threshold.
Now suppose that a field of interest is the top field A(1) in the first frame shown in FIG. 28, the two-field late field is the top field B(1) in the second frame. The differential value therebetween is above the predetermined threshold. When a field of interest is the top field B(1) in the second frame, the two-field late field is the top field B(1) in the third frame, and the differential value therebetween is ideally 0, namely below the predetermined threshold value.
If the result of comparison of the differential value with the predetermined threshold value determines that the differential value is equal to or smaller than the predetermined threshold value, the subsequent field that follows two fields behind the field of interest is detected as a repeat field.
As described above, in the 2-3 pulldown processed image signal, the repeat field appears every five fields in principle. Once the repeat field is detected, the subsequent field that follows five fields behind the repeat field is handled as a new field of interest. The differential value to the new field of interest is compared with the predetermined threshold value. When the differential value is equal to or smaller than the threshold value, the two-field later field is considered as a repeat field.
If the 2-3 pulldown processed image signal is subjected to editing process, for example, the repeat field may fail to appear every five fields. In such a case, even if the subsequent field that follows five fields behind the field of interest is handled as a new field of interest, the differential value of the new field is neither equal to nor smaller than the predetermined threshold value. The repeat field is difficult to detect if the five-field late field is handled as a new field of interest in this case.
When the five-field late field, handled as a new field of interest, has a differential value neither equal to nor smaller than the predetermined threshold, the immediately subsequent field (which follows one field behind the field of interest) is handled as a new field of interest, and the differential value of the new field of interest is compared with the predetermined threshold value. This process of handing the immediately subsequent field as a new field of interest, step by step, is repeated until a field having a differential value equal to or smaller than the predetermined threshold value appears. Hereinafter, this process is referred to as a step-by-step process.
When a field of interest having a differential value equal to or smaller than the predetermined threshold value and when the field of interest is odd number fields late from the last detected repeat field, the subsequent field that follows two fields behind the field of interest is detected as a repeat field. The subsequent field that follows five fields behind is handled as a new field of interest. Hereinafter, this process is referred to as a periodic process.
When a field of interest having a differential value equal to or smaller than the predetermined threshold value and when the field of interest is even number fields late from the last detected repeat field, the step-by-step process is repeated.
FIG. 29 illustrates a (2-3 pulldown processed) image signal in which an editing performed between a fifth frame and a sixth frame causes the appearance pattern of the repeat field to change from the duration of zero-th frame to fifth frame to the duration of sixth frame thereafter.
According to the detection method of the repeat field, the top field B(1) in the second frame is detected as a repeat field in the step-by-step process. The periodic process is then entered, in which the bottom field D(2) in the fourth frame is detected as a repeat field. The top field G(1) in the seventh frame that follows five fields behind the bottom field D(2) is not a repeat field and is not detected. The step-by-step process is thus initiated, and the top field G(1) in the eighth frame is detected as a repeat field. Control is shifted into the periodic process, in which the bottom field I(2) in the tenth frame that is five fields away is detected as a repeat field. Likewise, a repeat field is repeatedly detected every five fields thereafter.
There are times when the sixth frame and its subsequent frames have consecutively more still picture portion than the images of zero-th frame to the fifth frame in FIG. 29. The differential value of each of the sixth frame and its subsequent frames is almost 0, namely smaller than a predetermined value. As a result, the top field G(1) in the seventh frame that is five fields away from the bottom field D(2) in the fourth frame (repeat field) may erroneously be detected as a repeat field, even though it is actually not.
In the coding process, the top field G(1) in the seventh frame may be considered as a repeat field of the top field F(1) in the sixth frame that is two fields earlier, and is thus deleted.
When nearly still pictures appear consecutively, an erroneous detection is repeated every five fields, and fields which are not actually a repeat field are deleted. If such images are decoded, the motion of the images looks unnatural.
The 2-3 pulldown processed image signal is sometimes subjected to filtering process such as "time filter" to remove noise or to give a natural looking feature. Such a filtering increases the differential value between the repeat field and the prior field that precedes two fields ahead of the repeat field (the prior field comes in two fields earlier in time, namely a past field). As a result, fields that are actually a repeat field may escape detection. In this case, the repeat field is coded without being deleted, thereby degrading coding efficiency.
The periodic process detects a repeat field every five fields in the above repeat field detection method as described above. If no repeat field is detected five fields later, the detection of a repeat field through the step-by-step process is expected, at best, seven fields later than the last detected repeat field. At worst, the detection of a repeat field must wait for the field that is thirteen fields away.
FIG. 30 illustrates a (2-3 pulldown processed) image signal in which an editing performed between a fourth frame and a fifth frame causes the appearance pattern of the repeat field to change from the duration of zero-th frame to fourth frame to the duration of fifth frame and its subsequent frames thereafter.
The bottom field D(2) in the fourth frame appears as a repeat field and the top field F(1) in the sixth frame then comes in as a repeat field. According to the above-described repeat field detection method, however, the top field G(1) in the seventh frame that comes in five fields later is handled as a repeat field candidate after the bottom field D(2) in the fourth frame is detected as the repeat field. The top field F(1) in the sixth frame thus escapes repeat field detection.
Since the top field G(1) in the seventh frame that comes in five fields later than the bottom field D(2) in the fourth frame is not a repeat field, the step-by-step process is thereafter entered. The top field J(1) in the eleventh frame is the first repeat field detected since the step-by-step process was activated.
The repeat field that appears first after the activation of the step-by-step process is actually the bottom field H(2) in the eighth frame. Since the bottom field H(2) is even number, eight fields away from the last detected bottom field D(2) in the fourth frame, it is not detected as a repeat field. As a result, the first detected repeat field in the step-by-step process is the top field J(1) in the eleventh frame that is odd number, 13 fields away from the bottom field D(2) in the fourth frame.
Although the top field F(1) in the sixth frame and the bottom field H(2) in the eighth frame are redundant data, they are coded, thereby reducing coding efficiency.
One technique is available in which the step-by-step process only is performed with no periodic process combined therewith. As already described, such a technique erroneously detects a field as a repeat field and deletes it if nearly still images consecutively appear.
There are times when the construction of progressive pictures after the repeat fields are detected and then removed is not necessarily appropriate.
More particularly, when the top field B(1) in the second frame, one of the repeat fields, is deleted in FIG. 30, the two prior fields, namely the immediately prior field and the field that preceds two fields ahead of the repeat field (the bottom field B(2) in the first frame and the top field B(1) in the first frame) constitute a progressive picture and are then coded.
Although the repeat field and the field that precedes two fields ahead of the repeat field are essentially the same image, a slight discrepancy arising from filtering process by a time filter may take place therebetween.
Rather than deleting the repeat field to construct the progressive picture of the immediately prior field and the field that precedes two fields ahead of the repeat field, the field that precedes two fields ahead of the repeat field is deleted, and the progressive picture is constructed of the repeat field and the field immediately prior to the repeat field. This arrangement sometimes presents an improved coding efficiency.